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This Summer Reading Camp Self-Study Guide was developed to help state-, district-, 
and school-based practitioners and stakeholders conduct self-studies of planning and 
implementation of state-required summer reading camp programs for grade 3 students 
who scored at the lowest level on the state reading assessment. In some states these 
students face potential retention in grade 3 if they cannot meet grade-level standards 
for reading through a good-cause exemption, applicable alternate assessment score, 
or portfolio of student work showing mastery of grade-level reading standards. This 
guide provides a template for data collection and guiding questions for discussion that 
may improve instruction and increase the number of students meeting the grade-level 
standard by the end of the summer reading camp. 


Introduction 


Several Regional Educational Laboratory (REL) Southeast states are implementing or planning to imple- 
ment summer reading camps for grade 3 students who have scored at the lowest level on the state reading 
assessment. Summer reading camps are required multiweek extensions of the school year during which 
reading intervention is provided. Instruction provided during the summer reading camp is designed to 
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help students meet the gradedevel standard through a portfolio of student work or satisfactory performance 
on an alternate assessment. 

This self-study guide provides a template for data collection and guiding questions for discussion that may 
improve instruction and increase the number of students meeting the grade-level standard by the end 
of the summer reading camp. A computerized Google Forms web-based version of the guide designed to 
help facilitators collect individual ratings electronically is available for download from the REL Southeast 
website (http://rel-se.fcrr.org/) or by contacting REL Southeast at rel-se@fsu.edu. 

Florida has required summer reading camps since 2002, and North Carolina and South Carolina enacted 
legislation requiring implementation of grade 3 summer reading camps by the summer of 2014- Mississippi 
has enacted legislation authorizing grade 3 summer reading camps to begin in the summer of 2015. This 
policy is also in place outside the REL Southeast Region, such as in Arizona, Connecticut, Iowa, and 
Missouri. In states where this policy is in place, students who have scored at the lowest level on the state 
reading assessment face potential retention in grade 3 if they cannot meet the grade-level standard for 
reading through a good-cause exemption, applicable alternate assessment score, or portfolio of student work 
showing mastery of grade-level reading standards. 

This guide will help state-, district-, and school-based practitioners and stakeholders conduct self-studies of 
planning and implementing state-required summer reading camp programs for grade 3 students. Self-study 
is a process of using a guide with predetermined focus areas and questions to collect, share, and discuss data 
with stakeholders, in this case teachers, reading coaches, school-based administrators, and state and district 
administrators and literacy supervisors knowledgeable in summer reading camp policies and implementa- 
tion. This process may help educators document current practices in implementing a specific policy and 
determine how to improve performance. 

The guide was developed in partnership with REL Southeast’s Improving Literacy Research Alliance. Feed- 
back from alliance members helped shape the development of this guide, which was piloted with a Florida 
school district and refined based on the pilot report. 

States, districts, and schools that are implementing or planning to implement summer reading camps may 
find this guide helpful as they consider which types of evidence to collect and which components may be 
important for later evaluation. 

Purpose and use of the self-study guide 

The purposes of the Summer Reading Camp Self-Study Guide are to help states, districts, and schools: 

• Gather baseline information to use in developing an implementation plan for summer reading 
camps. 

• Prioritize their needs as they prepare to develop their implementation plan for summer reading 
camps. 

• Gather progress-monitoring information for continuous improvement of summer reading camps. 

• Evaluate the rollout and progress of summer reading camps. 

The most important part of this process for states, districts, and schools is the discussion that goes into the 
self-study. The scores on the Consensus Rating Form (see below) should reflect this facilitated discussion. 

This guide was designed to promote reflection about current strengths and challenges in planning or imple- 
mentation, spark conversations among staff, and identify areas for improvement. Based on pilot use of the 
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guide, this process will take about five hours, and time estimates are provided in the process steps outlined 
below. It is helpful to elicit input from participating teachers, literacy coaches, and schoohbased administra- 
tors in addition to state and district administrators and literacy supervisors. 

The self-study guide works best if a dedicated facilitator leads the effort. This facilitator should be knowb 
edgeable in best practices from research as well as in summer reading camp policies, procedures, and imple- 
mentation and should review the guide in detail before the selfistudy. The facilitator should also collect 
relevant data and possible sources of evidence before convening a meeting. The facilitator should be a 
careful listener and able to lead and structure discussions around evidence collection and decisionmaking 
processes for members of the self-study guide team. 

Components of the guide 

The Summer Reading Camp Self-Study Guide consists of the Scoring Guide, Consensus Rating Form, and 
Planning Next Steps Form. 

Scoring Quide. The Scoring Guide includes guiding questions and potential sources of evidence to support 
states and districts in reviewing state, district, and school-based planning and implementation of the camps. 
The Scoring Guide is tied to district and school actions and uses a four-point scale to assess the current 
status of implementation. The content of the Scoring Guide is based on the following eight areas, which 
research shows as being associated with strong implementation of summer reading camps: teacher effec- 
tiveness and qualifications; student selection and enrollment; instructional time; content and instruction; 
assessment selection and data use; mentoring and paraprofessional use; camp and classroom environment; 
and communication with administration, staff, and parents. Citations for each area are listed in the section 
“Support for Scoring Guide areas.” Box 1 lists the steps for completing the Scoring Guide. 

Consensus Rating Form. After the Scoring Guide is completed, a facilitator guides the self-study team 
through a consensus rating process. The team uses the Consensus Rating Form to reach agreement on the 
current status of implementation in the district or school and on planning the next steps. Box 1 lists the 
steps for completing the Consensus Rating Guide. 

Planning Next Steps Form. Prioritize the areas based on the strength of evidence and importance for 
success as described in the literature. Review the consensus scores showing a need to develop or improve. 
Identify two or three top priorities from the eight areas for action planning and record the priority areas. 
Complete a detailed plan for next steps and activities and note any potential challenges on the Planning 
Next Steps Form. [Activity length - 60 minutes] 
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Box 1. Steps to complete the Scoring Guide and Consensus Rating Form 

• Recruit the members who will make up the self-study team and convene a meeting to com- 
plete this process. Select a dedicated and knowledgeable facilitator. Then recruit teach- 
ers, reading coaches, school-based administrators, and district personnel knowledgeable 
in summer reading camp policies and implementation to complete the team. 

• Present an overview of the self-study process to all members of the team, including a 
review of relevant data and possible sources of evidence collected by the facilitator. 
[Activity length - 30 minutes] 

• Review, individually, the content of the Scoring Guide for each specific area that will be 
rated (for example, Teacher Effectiveness and Qualifications, Student Selection and Enroll- 
ment, and Instructional Time) and the section “Support for Scoring Guide areas.” 

[Activity length - 20 minutes] 

• Discuss any questions asked during the review. Questions should be answered by the 
facilitator after the overview and document review. 

[Activity length - 20 minutes] 

• Rate each area individually using the full Scoring Guide, including a review of relevant data/ 
possible sources of evidence provided by the facilitator. Each team member should rate 
each area independently; this is an opportunity for each person’s voice to be heard. A 
team member who does not know how to rate a specific area may abstain from rating it. 
[Activity length - 60 minutes] 

• Vote as a group to reach consensus. There are several steps to consensus voting 
[Activity length - 90 minutes]: 

• Vote. Ask each member to provide their numerical ranking for each area (1-8). 

• Identify frequency. Identify the most frequent number (if three members vote 3, five 
vote 2, and two vote 1, the most frequent number that members voted is 2). 

• Discuss rationale of the high frequency number. Ask someone who selected the con- 
sensus number to talk about what motivated that vote. 

• Discuss rationale of lower frequency numbers. Ask other members talk about why 
they voted in a particular way. 

• Vote. Use numeric voting a second time. Members may change their votes based on 
the discussion. 

• Record rating. If there is consensus, record the high frequency number on the Con- 
sensus Rating Form. If consensus is not reached (there is no high frequency number), 
continue discussing and voting until consensus is reached. 

• Continue across all areas. Repeat this process for each area. 

• Discuss and record initial team thoughts on priorities, next steps, and activities on the 
Consensus Rating Form. 

[Activity length - 20 minutes] 
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Self-Study Guide 


Scoring Guide 

Scoring Guide Area 1: Teacher Effectiveness and Qualifications SG-1 

Scoring Guide Area 2: Student Selection and Enrollment SG-2 

Scoring Guide Area 3: Instructional Time SG-3 

Scoring Guide Area 4: Content and Instruction SG-4 

Scoring Guide Area 5: Assessment Selection and Data Use SG-5 

Scoring Guide Area 6: Mentoring and Paraprofessional Use SG-6 

Scoring Guide Area 7: Camp and Classroom Environment SG-7 

Scoring Guide Area 8: Communication with Administration, Staff, and Parents SG-8 

Consensus Rating Form SG-9 

Planning Next Steps Form SG-10 

Support for Scoring Guide areas SG-11 


Scoring Guide Area 1: Teacher Effectiveness and Qualifications 

Policies are in place for hiring, developing, and retaining the best possible summer reading camp teachers for the 
populations of students served. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. The district/school has established criteria for hiring teachers with evidence of high performance Implementation progress 
in teaching reading to struggling readers, as indicated by their ability to engage students 
successfully (evaluations) and increase student performance (assessment data and student 
work samples). Teachers should also have a deep understanding of the materials/technology to 
be used (evaluations) (Borman & Dowling, 2006). Teachers should have specialized knowledge 

4 3 2 

in the alphabetic principle of how letters map to sounds in English, coupled with construction 
of meaning at the word, sentence, and text level (alphabetic principle, decoding and encoding 
including sound/spelling patterns, word study, vocabulary/word meaning, comprehension, oral 
language, critical thinking, and self-confidence) (Foorman & Wanzek, in 2015). 

Possible sources of evidence Guiding questions 


• District/school criteria for hiring and methods for recruiting effective 
teachers. 

■ Analysis of teacher data/performance evaluations. 

• Review of teacher certification/endorsements. 

• Memos to schools. 

• Teachers’ applications. 


Has the district/school established criteria 
for hiring teachers with evidence of high 
performance in teaching reading to at-risk 
grade 3 students for summer reading camps? 

Has the district/school considered components 
of effectiveness (including a review of any 
available data) during the hiring process? 

Are successful summer reading camp teachers 
from prior years recruited for subsequent years? 


2. The district/school provides professional development in research-based instructional practices Implementation progress 
and support that will ensure development and retention of the best possible summer reading 
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camp teachers for the populations of students served (Kim & Quinn, 2013). 


Possible sources of evidence 

• Professional development plans for summer reading camp 
teachers and paraprofessionals, including instructional materials, 
plans for maximizing the instructional schedule, instructional 
practices empirically shown to affect gains in student 
achievement, and data intended to inform instruction. 

• Plans and logs of embedded coaching support/follow-up training 
from prior summer reading camps. 

• Documentation of administrative walk-throughs and support for 
improving the classroom environment and instructional practices. 

• Interviews with administrators and staff. 


Guiding questions 

• Does the professional development offered to summer 
reading camp teachers and paraprofessionals build 
the knowledge they need for success and provide 
differentiated support based on their needs? 

■ Are summer reading camp teachers offered follow-up 
training or embedded coach support? 

• Do district/school leaders provide support to summer 
reading camp teachers? 


• Documentation of coaching and differentiated support. 

• Professional development records. 

• Professional development evaluations. 
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Scoring Guide Area 2: Student Selection and Enrollment 

Parents of students showing deficiencies in reading during the school year are notified in a timely manner. Eligible students 
are successfully enrolled, and parents are provided accurate, timely information about student selection and attendance. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. The district/school has a policy for early notification for parents of students showing deficiencies 

Implementation progress 

in reading during the school year. Students scoring below the state proficiency level in reading 
are quickly identified once reading scores are released to determine eligibility for summer reading 
camp (Hamilton et al., 2009). 

4 
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Possible sources of evidence Guiding questions 


■ Strategic plan. 

• Memos to schools. 

• Diagnostic/progress monitoring data through the response to 
intervention process. 

• Parent notification sample letters/templates. 

■ Student portfolios. 

• Administrative/guidance counselor/assessment coordinator training. 


Is there a process to ensure that educators 
are notifying parents about potential reading 
deficiencies during the school year? 

Is there a process to ensure efficient 
data collection/reporting from the reading 
assessment once scores are released? 


2. Procedures are in place to provide summer reading camp parents with accurate, timely Implementation progress 
information about all relevant policies, procedures, and expectations for attendance (Borman & 
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Dowling, 2006). 


Possible sources of evidence 

■ Strategic plan. 

• Memos to schools. 

• Parent notification sample letters/templates. 

• Parental support documentation. 

■ Notification to charter schools inviting eligible grade 3 students 
to attend district camps. 

■ Attendance policy, records, and notifications. 


Guiding questions 

• Does the district have timelines/guidelines for parental 
notification? 

■ Are there expectations for schools/teachers to 
communicate with parents during the summer reading 
camp? 
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Scoring Guide Area 3: Instructional Time 

Summer reading camp schedules facilitate intensive reading intervention for the populations of students served. The district/ 
school motivates student attendance and engagement. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. The district/schoo! has established a schedule that will maximize student instructional time Implementation progress 
(Foorman & Al Otaiba, 2009). The district/school motivates student attendance and engagement 

(Borman & Dowling, 2006). 4 3 2 1 

Possible sources of evidence Guiding questions 


School and district plans. 

Staff, student, and parent interviews. 

Student attendance records. 

Published schedules by the district/schools, including the number of 
days, weeks, and hours of instruction (Kim & Quinn, 2013). 

Reviews of student reading data, summer reading camp calendars, and 
instructional schedules that align instructional time and student needs. 


• Do instructional schedules created by the 
district/schools allow time for whole group (13 or 
fewer students) and small group (3-5 students) 
instruction? 

• Does the length of time students participate 
in the camp offer the optimal amount of 
instructional time for positive outcomes 
(Schacter & Jo, 2005)? 

• Does the district/school consider ways to 
motivate student attendance and engagement? 


2. Students receive as much small group (3-5 students) instructional time as possible (Shanahan 
et al., 2010). 


Implementation progress 
4 3 2 1 


Possible sources of evidence 


Guiding questions 


School plans. 

Classroom schedules. 

Differentiated schedules providing more time for English language 
learners, special education students, and other students with severe 
reading difficulties. 

Staff interviews. 

District observations. 


Does small group instruction occur on an 
ongoing basis? 

Is the amount of small group instructional time 
provided differentiated based on specific student 
needs? 

Does the district/school provide professional 
support to staff seeking assistance in 
structuring small group instructional time? 
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Scoring Guide Area 4: Content and Instruction 

The design of the reading curriculum and the plan for reading instruction and intervention reflect instructional practices 
that have been empirically shown to support gains in student achievement. The focus should be on explicit and systematic 
instruction in foundational reading skills based on student need (for example, phonics and vocabulary/oral language 
comprehension). 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. The district/school has established criteria for selecting and using reading programs and Implementation progress 
curricula that have been shown to have a positive effect on student achievement (Coalition for 

Evidence-Based Policy, 2003). 4 3 2 1 

Possible sources of evidence Guiding questions 


• Documentation of program selection criteria with an emphasi: 
materials that have rigorous research support. 

• Policies and procedures for adoption of materials. 

■ Analysis of program use and classroom observations. 

• Memos to schools. 

• Professional development records. 


on using • Have criteria been developed to select programs 
and materials for use with students in summer 
reading camps? 

• Has professional development been provided 
to summer reading camp teachers to support 
effective use of reading programs and 
curriculum? 


2. The district/school has implemented a plan for reading instruction and intervention that reflects 

Implementation progress 

instructional practices empirically shown to increase student achievement. The focus is on 
explicit and systematic instruction in foundational reading skills based on student need (for 
example, phonics and vocabulary/oral language comprehension) (Gersten et al., 2008). 

4 

3 

2 
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Possible sources of evidence Guiding questions 


Professional development plans for summer reading camp teachers 
including instructional materials (Borman & Dowling, 2006), an 
instructional schedule that maximizes instructional time, and 
instructional practices empirically shown to increase student 
achievement. 


Does the professional development offered 
for summer reading camp teachers focus on 
instructional practices empirically shown to 
increase student achievement (Beck & Beck, 
2012 )? 


Instructional plans created by district staff and teachers. 
Interviews with administrators and staff. 

Classroom observations. 

Professional development records. 

Professional development evaluations. 

Progress monitoring tools and data. 


Do district/school staff help teachers develop 
instructional plans for reading instruction/ 
intervention using student assessment 
information? 

Do classroom observations help district/school 
staff verify implementation and support of 
effective instructional practices? 

Is there research evidence of efficacy for 
teachers' instructional practices with regard to 
student outcomes? 
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Scoring Guide Area 5: Assessment Selection and Data Use 

Exemptions from the reading requirement are approved by the state education agency for use in making promotion decisions. 
Other valid and reliable standardized assessments are selected based on their alignment with instructional content and are 
used to establish small groups and inform instruction. (This may include assessments that provide information on student 
performance in fluency, word knowledge, vocabulary, and comprehension, for example.) 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. Only exemptions from the requirement approved by the state education agency are used in 
making promotion decisions. 


Implementation progress 
4 3 2 1 


Possible sources of evidence 


Guiding questions 


• Documentation of assessments used for promotion decisions. 

• Documentation of a review of state policies and procedures in place for 
exemptions from the state requirement. 

• Records of students promoted through state-approved exemptions. 

■ Student portfolios. 


Does the district/school verify policies and 
procedures for state-approved exemptions used 
for promotion? 

Does the district/school maintain accurate 
records for students promoted through state- 
approved exemptions? 


2. Other valid and reliable standardized assessments (including diagnostic reading assessments) Implementation progress 
were selected based on alignment with the content of instruction (such as performance in fluency, 

word knowledge, vocabulary, and comprehension) and were used to establish small groups and 4 3 2 1 

inform instruction (Hamilton et al., 2009). 

Possible sources of evidence Guiding questions 


Documentation of reading assessment selection criteria. 

Documentation of alignment between reading assessments and 
instructional materials. 

Professional development plans for summer reading camp teachers that 
include assessment policies, data-based instructional decisionmaking, 
and work with small student groups based on instructional needs. 


Instructional plans created by district staff and teachers. 
Interviews with administrators and staff. 

Classroom observations. 

Professional development records. 

Professional development evaluations. 

Data review from the end of summer reading camps. 
Student portfolios. 


Do valid and reliable assessments provide 
information on student performance in 
fluency, word knowledge, vocabulary, and 
comprehension? 

Does the professional development offered 
for summer reading camp teachers focus on 
assessment policies, data-based instructional 
decisionmaking, and work with small student 
groups based on instructional needs? 

Do classroom observations help district and 
school staff verify and support effective data- 
based instructional decisionmaking and small 
group instruction based on student needs? 

From a review of data following the conclusion 
of summer reading camps, do results show that 
students are improving key reading skills as a 
result of the instruction? 

Is planning for future summer reading camps 
based on student data from previous summer 
reading camps? 
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Scoring Guide Area 6: Mentoring and Paraprofessional Use 

The district or school develops and implements a plan to provide trained mentors and paraprofessionals to reinforce 
students’ reading skills in the smallest group sizes possible. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. The district or school develops and implements a plan to provide trained paraprofessionals, Implementation progress 
if available, to reinforce students' reading skills in the smallest group sizes possible (Elbaum, 

Vaughn, Tejero Hughes, & Watson Moody, 2000). 4 3 2 1 

Possible sources of evidence Guiding questions 


• Documentation of hiring, training, and work hours of paraprofessionals. 

• Documentation of summer reading camp schedules providing 
instructional time in small groups to reinforce student reading skills. 


Does the district/school hire, train, and use 
paraprofessionals to reduce group sizes and 
reinforce student reading skills in summer 
reading camps? 


2. The district plans to use mentors to reinforce students' reading skills in the smallest group sizes Implementation progress 
possible (Foorman & Al Otaiba, 2009). 4 3 2 1 


Possible sources of evidence 


Guiding questions 


• Documentation of recruitment, training, and volunteer hours of mentors. 

• Documentation of summer reading camp schedules providing 
instructional time in small groups to reinforce student reading skills. 


Does the district/school recruit, train, and use 
mentors to reduce group sizes to help reinforce 
student reading skills in summer reading camps? 
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Scoring Guide Area 7: Camp and Classroom Environment 

A healthy and safe learning environment is established that is conducive to student engagement and intensive reading 
instruction. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. A healthy and safe learning environment has been established. 


Possible sources of evidence 

• Documentation of district/school criteria for school/classroom 
selection for summer reading camp use. 

• Documentation of a custodial/maintenance plan for schools/ 
classrooms during summer reading camp use. 

• Procedures are established for teachers to report concerns about the 
learning environment and for these concerns to be addressed quickly. 


Implementation progress 
4 3 2 1 

Guiding questions 

■ Have criteria been developed to select schools/ 
classrooms for summer reading camps that will 
provide healthy and safe learning environments 
free from maintenance distractions (painting, 
floor waxing, and other work)? 

■ Do classroom observations help district/school 
staff verify selection of schools/classrooms that 
provide a healthy and safe learning environment? 


2. The learning environment is conducive to student engagement and intensive reading instruction Implementation progress 
(Tanner, 2008). 4 3 2 1 

Possible sources of evidence 


Documentation of media center availability and support at the summer 
reading camp site. 

Documentation of technology availability and support at the summer 
reading camp site. 

Documentation that instructional materials and other needed resources 
are available and organized at the summer reading camp site. 

Documentation of a selection process for schools/classrooms that 
includes the best physical environment for summer reading camps. 

Interviews with administrators and staff. 

Classroom observations for student engagement and motivation. 

Class size. 

Documentation of print-rich environments. 


Guiding questions 

■ Is the media center at the summer reading camp 
site open and staffed? 

■ Is all technology needed for summer reading 
camp programs available, with support staff as 
needed? 

• Are instructional materials and any other needed 
resources for summer reading camp teachers 
available and organized at the summer reading 
camp site? 

■ Do classroom observations help district/ 
school staff verify that the summer reading 
camp environment is conducive to student 
engagement, motivation, and intensive reading 
instruction? 

■ Do selected schools/classrooms include the 
best physical environment for summer reading 
camps (including space and furniture for large 
and small group instruction, lighting, and other 
features)? 
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Scoring Guide Area 8: Communication with Administration, Staff, and Parents 

A communication plan is developed and implemented to provide relevant information for summer reading camp administrators 
and staff and for ongoing communication with students' prior-year and next-year teachers and with parents to help reinforce 
students’ reading skills at home. 

4 = Already in place 

Circle the rating that best describes your program's 3 = Partially in place, under development 
implementation progress for each item. 2 = Area to develop or improve 

1 = Not feasible now 


1. A communication plan was developed and is being implemented to provide relevant information Implementation progress 
for summer reading camp administrators and staff and for ongoing communications with 

4 3 2 1 

students' prior-year and next-year teachers (Hamilton etal., 2009). 

Possible sources of evidence Guiding questions 


Documentation of a district/school communication plan for 
administrators and staff providing instruction during summer reading 
camps. 


Does the district/school provide timely, 
accurate, and relevant information to summer 
reading camp administrators and staff? 


• Documentation of communication between summer reading camp 
teachers and prior grade 3 teachers and next-year teachers. 

• Memos from the district to schools. 

• Memos from school administration to teachers. 


Do summer reading camp teachers communicate 
with prior-year and next-year teachers to obtain 
and share information that will support data- 
based decisionmaking? 


• Interviews with administrators and staff. 


2. A communication plan was developed and implemented to provide ongoing communication with 
parents to reinforce students’ reading skills at home (Gonzalez-DeHass, Willems, & Holbein, 
2005). 


Implementation progress 
4 3 2 1 


Possible sources of evidence 

• Documentation of communication with parents. 

• Documentation of activities provided for parents to help reinforce 
students’ reading skills (Kim & Quinn, 2013). 

• Documentation of ongoing parental support for student reading 
instruction (Henderson & Mapp, 2002). 

• Interviews with administrators and staff. 


Guiding questions 

■ Does the district/school have a plan to establish 
and maintain communication with parents? 

• Do teachers provide students and parents with 
activities and information that will help reinforce 
students’ reading skills at home? 


• Interviews with parents. 

■ Summer reading camp progress reports and student activities for 
parents. 
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Consensus Rating Form 

(to be completed by the facilitator) 


State 


District 

School 


Complete this form by recording the results of consensus ratings and discussions from initial self-study 
results, initial thoughts on priorities, and initial brainstorming ideas for next steps/activities for each area 
rated 2 or 3. 


Scoring Guide area Consensus 9 Priorities Ideas for next steps/activities 


Area 1. Teacher Effectiveness and Part 1: 4 3 2 1 

Qualifications Part 2: 4 3 2 1 

Area 2. Student Selection and Enrollment Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 3. Instructional Time Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 4. Content and Instruction Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 5. Assessment Selection and Data Use Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 6. Mentoring and Paraprofessional Use Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 7. Camp and Classroom Environment Part 1: 4 3 2 1 

Part 2: 4 3 2 1 

Area 8. Communication with Administration, Part 1: 4 3 2 1 

Staff, and Parents Part 2: 4 3 2 1 


a. 4 = Already in place; 3 = Partially in place, under development; 2 = Area to develop or improve; 1 = Not feasible now. 
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Planning Next Steps Form 

(to be completed by the facilitator) 


After the Consensus Rating Form has been completed, the facilitator will begin the completion of this 
form by leading a discussion with the group about the priorities for action based on the strength of evi- 
dence and importance for success as described in the literature. The facilitator will next lead a discussion 
for the development of a detailed plan for next steps and activities that are most urgent and actionable. 
Finally, the facilitator will lead a discussion to capture potential challenges to the plan. 


Based on group discussion and consensus ratings, list the top priority areas to develop or improve. 


Based on group discussion, what next steps and activities are needed to address the listed priorities? Consider timelines and 
who will be responsible for determining the strategies or providing the resources. 


Based on group discussion, what general challenges do you anticipate? How will the challenges be addressed? Consider who 
will be responsible for addressing these challenges. 
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Support for Scoring Guide areas 


This appendix describes key references that provide additional support for each of the Scoring Guide areas. 

Scoring Guide Area 1: Teacher Effectiveness and Qualifications 

Borman, G. D., & Dowling, N. M. (2006). Effects of multiyear summer school: Evidence from the Teach 
Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48. 

In this work, Borman and Dowling note the importance of expectations and training for teachers hired for 
the Teach Baltimore Summer Academy program. Teachers hired for this program received three weeks of 
preservice training in reading curricula, lesson planning, and classroom management. Also noted was that, 
“Aligning the content of the program with the regular schoohyear materials and instruction used in the 
schools served by the program also helped convince parents, teachers, and principals of the saliency of the 
Summer Academy for students’ continued academic success during the school year” (p. 45). 

Foorman, B., & Wanzek, J. (2015). Classroom reading instruction for all students. In S. R. Jimerson, M. K. 
Burns, & A. M. VanDerHeyden (Eds.), The handbook of response to intervention: The science and practice 
of multi-tiered systems of support. New York: Springer Science. 

In this work, Foorman and Wanzek note: “What is abundantly clear from research (e.g., Foorman, Francis, 
Fletcher, Schatschneider, & Mehta, 1998; Mathes et al., 2005; Simmons et al., 2008) and from consensus 
documents (National Research Council, 1998; National Institute of Child Health and Human Develop- 
ment, 2000) is that explicit instruction in the alphabetic principle of how letters map to sounds in English 
is necessary to learn to decode and to prevent reading difficulties. However, mastery of the alphabetic prin- 
ciple must be coupled with construction of meaning — at the word, sentence, and text level — if compre- 
hension is to occur (Foorman & Connor, 2001; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2001). 
Thus, reading/language arts instruction in the primary grades must consist of (a) mastering the encoding 
and decoding of the alphabet and (b) building oral and academic language and written language compre- 
hension (American Educational Research Association, 2009)” (p. 10). 

Kim, J. S., & Quinn, D. M. (2013). The effects of summer reading on low-income children’s achievement 
from kindergarten to grade 8: A meta-analysis of classroom and home interventions. Review of Educa- 
tional Research, 83(3), 386-431. 

In this meta-analysis, Kim and Quinn note that with summer “classroom interventions, the quantity and 
quality of teacher-directed literacy instruction is the critical mechanism that promotes reading achieve- 
ment (Tseng & Seidman, 2007)” (p. 387). In addition, Kim and Quinn note the need to “illuminate the 
critical mechanisms inside classrooms — most notably, the quality of teachers’ instructional practice and 
emotional support for learning — that underlie the observed improvements in reading achievement during 
the summer” (p. 35). Kim and Quinn found that “there was a positive impact of classroom interventions 
using research-based instruction on reading comprehension total” (p. 410). Also noted is that “class- 
room-based interventions should implement explicit, teacher-directed instruction of high-utility words that 
enable children to read proficiently during the school year (Beck & McKeown, 1991; Snow, 2002)” (p. 419). 
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Scoring Guide Area 2: Student Selection and Enrollment 


Borman, G. D., & Dowling, N. M. (2006). Effects of multiyear summer school: Evidence from the Teach 
Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48. 

Evidence from this study suggests that “developing a better understanding of how parents and schools from 
high'poverty communities can work together to improve participation in summer school could prove to be 
a highly productive research and policy initiative” (p. 46). 

Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student 
achievement data to support instructional decision making (NCEE No. 2009-4067). Washington, DC: 
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evalua- 
tion and Regional Assistance, http://ies.ed.gov/ncee/wwc/publications/practiceguides/ 

Recommendation 3 from this practice guide suggests that educators should establish a clear vision for 
schoolwide data use. They can do so by developing “a strong culture of data use to ensure that data-based 
decisions are made frequently, consistently, and appropriately. This data culture should emphasize collabo- 
ration across and within grade levels and subject areas to diagnose problems and refine educational practic- 
es. Several factors (e.g., planning, leadership, implementation, and attitude) affect the success schools will 
have with developing and maintaining a data culture” (p. 27). 

Scoring Guide Area 3: Instructional Time 

Borman, G. D., & Dowling, N. M. (2006). Effects of multiyear summer school: Evidence from the Teach 
Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48. 

Evidence from this study suggests that summer school programs can “help improve longitudinal learning 
for students from high-poverty schools, but only with students’ regular attendance in the program” (p. 46). 

Foorman, B., & A1 Otaiba, S. (2009). Reading remediation: State of the art. In K. Pugh & P. McCardle 
(Eds.), How children learn to read (pp. 257-274). New York: Psychology Press. 

Foorman and A1 Otaiba conclude: “The principal’s leadership is crucial to guaranteeing a minimum of 90 
minutes a day of uninterrupted reading/language arts instruction, with flexible scheduling to allow supple- 
mental small-group instruction that addresses the needs of individual students not responding adequately 
to classroom instruction. Schools as well as entire districts (e.g., Fielding, Kerr, & Rosier, 2007) have imple- 
mented a successful RTI model based on these principles” (p. 270). 

Kim, J. S., & Quinn, D. M. (2013). The effects of summer reading on low-income children’s achievement 
from kindergarten to grade 8: A meta-analysis of classroom and home interventions. Review of Educa - 
tional Research, 83(3), 386-431. 

Kim and Quinn find that “[sjtudies that provide fewer than 43 hours or more than 210 hours did not 
produce statistically significant, positive effects” (p. 424). In addition this meta-analysis found, “suggestive 
evidence that the five resource-intensive programs with small class sizes of 13 or fewer children, 4 to 8 hours 
of daily program time, and 70 to 175 hours of total program time had a positive effect on reading achieve- 
ment” (p. 420). 
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Schacter, J., & Jo, B. (2005). Learning when school is not in session: A reading summer day-camp inter- 
vention to improve the achievement of exiting first-grade students who are economically disadvan- 
taged. Journal of Research in Reading, 28(2), 158-169. 

This work suggests “that the duration of the majority of summer school programmes, often between three 
and six weeks, is too short for students to accomplish meaningful learning gains” (p. 160). 

Shanahan, T., Callison, K., Carriere, C., Duke, N. K., Pearson, P. D., Schatschneider, C., & Torgesen, J. 
(2010). Improving reading comprehension in kindergarten through 3rd grade: A practice guide (NCEE No. 
2010-4038). Washington, DC: U.S. Department of Education, Institute of Education Sciences, Nation- 
al Center for Education Evaluation and Regional Assistance. 

This practice guide for improving reading comprehension for kindergarten to grade 3 students recommends 
that teachers “form small groups of students with similar comprehension needs or skills, allowing them to 
focus targeted help on a few students at a time” (p. 16). Also, teachers should “[h]ave students lead struc- 
tured small-group discussions” (p. 27). 

Scoring Guide Area 4: Content and Instruction 

Beck, I. L., & Beck, M. E. (2012). Making sense of phonics: The hows and whys (2nd ed.). New York: Guilford 
Publications. 

In this book, Beck and Beck point out the evidence that teacher understanding of systematic and explicit 
phonics instruction is key to student success in reading. 

Borman, G. D., & Dowling, N. M. (2006). Effects of multiyear summer school: Evidence from the Teach 
Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48. 

This study suggests that participating educators in the Teach Baltimore Summer Academy program begin 
with three weeks of preservice training, including training in reading curricula. 

Coalition for Evidence-Based Policy. (2003). Identifying and implementing educational practices supported by 
rigorous evidence: A user-friendly guide. Washington, DC: U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Evaluation and Regional Assistance. 

According to this guide, “If practitioners have the tools to identify evidence-based interventions, they may 
be able to spark major improvements in their schools and, collectively, in American education” (p. iii). 

Gersten, R., Compton, D., Connor, C. M., Dimino, J., Santoro, L., Linan-Thompson, S., & Tilly, W. D. (2008). 
Assisting students struggling with reading: Response to intervention and multi-tier intervention for reading in the 
primary grades. A practice guide (NCEE No. 2009-4045). Washington, DC: U.S. Department of Educa- 
tion, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. 

This practice guide finds strong evidence for providing “intensive, systematic instruction on up to three 
foundational reading skills in small groups to students who score below the benchmark score on universal 
screening. Typically, these groups meet between three and five times a week, for 20 to 40 minutes” (p. 6). 
This guide also recommends that teachers “[pjrovide intensive instruction on a daily basis that promotes 
the development of the various components of reading proficiency to students who show minimal progress 
after reasonable time in tier 2 small group instruction (tier 3)” (p. 6). 
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Scoring Guide Area 5: Assessment Selection and Data Use 


Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student 
achievement data to support instructional decision making (NCEE No. 2009-4067). Washington, DC: 
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evalua- 
tion and Regional Assistance, http://ies.ed.gov/ncee/wwc/publications/practiceguides/ 

Recommendation 1 of this practice guide notes: “Teachers should adopt a systematic process for using data 
in order to bring evidence to bear on their instructional decisions and improve their ability to meet stu- 
dents’ learning needs. The process of using data to improve instruction, the panel believes, can be under- 
stood as cyclical. It includes a step for collecting and preparing data about student learning from a variety 
of relevant sources, including annual, interim, and classroom assessment data. After preparing data for 
examination, teachers should interpret the data and develop hypotheses about factors contributing to stu- 
dents’ performance and the specific actions they can take to meet students’ needs. Teachers then should 
test these hypotheses by implementing changes to their instructional practice. Finally, they should restart 
the cycle by collecting and interpreting new student performance data to evaluate their own instructional 
changes” (p. 10). 

Scoring Guide Area 6: Mentoring and Paraprofessional Use 

Elbaum, B., Vaughn, S., Tejero Hughes, M., & Watson Moody, S. (2000). How effective are one-to-one 
tutoring programs in reading for elementary students at risk for reading failure? A meta-analysis of the 
intervention research. Journal of Educational Psychology, 92(4), 605-619. 

This meta-analysis concludes that “[t]he effectiveness of one-to-one instruction has been validated by 
empirical research, especially for students who are considered at risk for school failure or have been iden- 
tified as having reading or learning disabilities” (p. 605). The meta-analysis also revealed that “college 
students and trained, reliable community volunteers were able to provide significant help to struggling 
readers” (p. 616). The use of paraprofessionals for similar purposes was also cited in the meta-analysis. 

Foorman, B., & A1 Otaiba, S. (2009). Reading remediation: State of the art. In K. Pugh & P. McCardle 
(Eds.), How children learn to read. (pp. 257-274) New York: Psychology Press. 

This book concludes: “Several decades of research on effective reading intervention shows us that it is 
possible to substantially reduce the 36% of fourth graders reading below grade level on NAEP or the 17.5% 
of students with reading disabilities. Better classroom instruction can reduce the numbers of low-achieving 
students to around 5%. Supplemental small-group or one-on-one tutoring can reduce the numbers even 
further to l%-3%” (p. 270). 

Scoring Guide Area 7: Camp and Classroom Environment 

Tanner, C. K. (2008). Explaining relationships among student outcomes and the school’s physical environ- 
ment. Journal of Advanced Academics, 19(3), 444-471. 

The summary of this descriptive study notes that the purpose was to investigate “the possible effects of 
selected school design patterns on third-grade students’ academic achievement” (p. 445). The school’s phys- 
ical environment was defined through four sets of design patterns: “movement and circulation (e.g., ade- 
quate personal space and efficient movement patterns throughout the school), large group meeting places 
(e.g., social gathering places), day lighting and views (e.g., windows with natural light), and instructional 
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neighborhoods (e.g., large and small group areas that accommodate wet and dry activities)” (p. 445). The 
findings note that “each of the four full regression models, which included subsets of the design elements, 
explained between 2% and 7% of additional variance in achievement when compared to the reduced 
model, which included a measure of school SES. Therefore, each of the four design variables was positively 
related to student achievement, even after controlling for school SES” (p. 445). 

Scoring Guide Area 8: Communication with Administration, Staff, and Parents 

Gonzalez'DeHass, A. R., Willems, R P., & Holbein, M. F. D. (2005). Examining the relationship between 
parental involvement and student motivation. Educational Psychology Review, 17(2), 99-123. 

This review reports that teacher involvement with parents at home helped reinforce student reading. It 
notes that parents in a study who received communication from the school and were encouraged to help 
their children with reading activities and listen to them read reported that “their child’s overall interest, 
enjoyment in reading, and motivation to read had increased. In fact, parents as well as teachers in the 
home-school conditions reported seeing students display more interest in discussing books, reading books 
not assigned, and being motivated to read at home” (p. 117). 

Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student 
achievement data to support instructional decision making (NCEE No. 2009-4067). Washington, DC: 
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evalua- 
tion and Regional Assistance, http://ies.ed.gov/ncee/wwc/publications/practiceguides/ 

Recommendation 3 of this practice guide suggests: “Schools must establish a strong culture of data use to 
ensure that data-based decisions are made frequently, consistently, and appropriately. This data culture 
should emphasize collaboration across and within grade levels and subject areas to diagnose problems and 
refine educational practices” (p. 27). 

Henderson, A. T., & Mapp, K. L. (2002). A new wave of evidence: The impact of school, family, and communi- 
ty connections on student achievement (Annual Synthesis 2002). Austin, TX: National Center for Family 
& Community Connections with Schools Southwest Educational Development Laboratory. 

A summary provided in this synthesis report notes: “When schools engage families in ways that are linked 
to improving learning, students make greater gains. When schools build partnerships with families that 
respond to their concerns and honor their contributions, they are successful in sustaining connections that 
are aimed at improving student achievement” (p. 8). 

Kim, J. S., & Quinn, D. M. (2013). The effects of summer reading on low-income children’s achievement 
from kindergarten to grade 8: A meta-analysis of classroom and home interventions. Review of Educa- 
tional Research, 83(3), 386-431. 

Kim and Quinn note: “Although most home interventions do not include a schoohbased event prior to the 
summer, it is critical to strengthen the home'School connection” (p. 421). 
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